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^ (57) Abstract: System and method for inspecting prints. An electronic processor is used to determine whether prints are charac- 

O terized by an image-quality level that falls outside a predetermined range. The electronic processor comprises an artificial neural 

^ network trained to recognize good prints and bad prints based on histograms, statistical abstractions or raw pixel data derived from 

^ digital images representing each print. 
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RELATED APPLICATIONS 

This application claims priority under 35 U.S.C. 1 19 to provisional 
application 60/150,075, filed August 20, 1999, the entirety of which is incorporated 
herein by reference. 



FIELD OF THE INVENTION 

The present invention relates generally to the field of photography, and 
more particularly to photographic film processing. Still more particularly, the present 
invention relates to the task of inspecting photographic prints for defects in exposure and 
image-quality and sorting out defective prints before they are sent to the customer. 



RELATED ART 

Photographic prints are typically inspected in photo finishing laboratories 
by trained operators who manually apply infrared absorbent stickers to prints that come 
out of the developing process blank, fogged or otherwise damaged beyond a level at 
which the print can be sold. In addition to problems caused by over and under exposure, 
photographic prints can have physical defects, such as scratches, water spots, edge 
damage or emulsion tears. Sometimes prints are damaged by repetitive exposure errors. 
These kinds of defects, which can be the result of a hair or film chip falling onto the 
exposure mechanism of the image printer, for example, can affect a large number of 
prints. 

Current methods for inspecting prints require human intervention. A 
human being must manually examine every single print to determine whether the print 
should be rejected. Unfortunately, photo processing is much more expensive and time- 
consuming when a human being is involved. For example, when a human inspects prints 
moving on an assembly line, the human must periodically stop the assembly line, in order 
to afford the time necessary to closely examine a particular print and make a decision 
about whether the print should be accepted or rejected. Thus, the whole photo processing 
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job is slowed down tremendously because of the human interaction, which causes the 
process to stop and start over and over again. 

Many human operators are trained in the art of recognizing defects. But 
the training itself is expensive and time consuming. Plus, an employer never knows 
whether he will see a return on training investment dollars because human beings tend to 
quit, retire, take other jobs, get sick, or otherwise stop performing the inspection task for 
which they have been trained. Moreover, no matter how well a human is trained in the 
art of inspecting prints, the decision to accept or reject a particular print is ultimately an 
entirely subjective decision. Thus, even for a group of highly trained individuals, results 
will be inconsistent, depending on which operator is doing the inspection. 

Another way of overcoming the problems with bad prints is to use very 
sophisticated printing equipment. Typically, these printers operate by scanning the image 
contained in a frame of film in a number of different ways to ascertain which bands of 
light are present as a result of incorrect exposure. These printers then either attempt to 
print the image in such a way as to overcome the defect, or simply skip images that 
contain defects printing adjustments cannot overcome. Unfortunately, there are some 
very significant drawbacks to using printers to identify and reject defective prints. 

First, the printers are extremely expensive. Second, they add an 
unacceptable amount of additional time to the photo finishing process. Third, they do not 
address defects that come about as a result of something that occurs during or after the 
image printing process, such as foreign objects falling in the optical path, physical paper 
damage, defective printing equipment, incorrect paper development and numerous other 
problems, some of which are not immediately obvious. And fourth, using more 
sophisticated printers has not been found to be as reliable for rejecting defective prints as 
using human beings, mainly because printers can only identify incorrect exposures within 
a very limited range. 

Accordingly, there is a need recognized by inventors in the photo 
processing industry for fast, consistent and economical method for detecting and rejecting 
prints that are defective without human intervention. 
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SUMMARY OF THE INVENTION 

The present invention is directed to an automated system and method for 
inspecting photographic prints, detecting defects in those prints, and removing the 
defective prints from the job. In one aspect of the present invention, a print inspection 
system is provided that comprises: a transport mechanism for moving a length of paper 
containing a plurality of prints along a work path; a scanner, positioned along the work 
path configured to capture a visual image of each print; means, coupled to the scanner, 
for converting the visual image of each print into a digital file corresponding to the vxsual 
im age- and an electronic processor that determines whether the visual image is 
characterized by an image-quality level that falls inside a predetermined image-quality 
ranee In the preferred embodiment, the electronic processor is comprised of an artificial 
neural network processor trained to recognize digital files having image-quality levels 
that fall outside the predetermined range. 

The print inspection station of the present invention can also comprise 
means for storing the digital files corresponding to the visual images contained on the 
prints in a storage medium for later processing. In a further preferred embodiment, the 
re suh of the image-quality level determination can also be stored in a storage medium. 

Although the above-summarized embodiments are directed to detecting 
,0 prints characterized by an image-quality level that falls outside a predetermined range, 
person of skill m me art would recognize that me same results could be achieved by 
alternate means, such as detecting prints characterized by image-quality levels that fall 
inside a predetermined range. Such alternate means are considered within the scope of 

the present invention. 

In another aspect of the present invention, prints determined to image 
qua litv levels that fall outside a predetermined range are marked as defective. Inyet 
another aspect of the present invention, prints so marked are automatically discarded. In 
still another aspect of the current invention, the print inspection system also includes 
means for tracking the current location of a print as it moves along the work path. This 
preferred embodiment also includes means for transmitting the image-quality level and 
current location of that print within the work path to another print processing device. 
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In still a further aspect of the present invention, a method or process for inspecting prints 

is provided. The method comprises the steps of: feeding into a work path a length of 
paper containing a plurality of prints; capturing a visual image of each print; converting 
the visual image of each print into a digital file corresponding to the visual image; 
determining whether the visual image is characterized by an image-quality level that falls 
outside a predetermined image-quality range; and repeating the determining step for each 
digital file created. In the preferred embodiment of this method, an electronic processor 
is used to make the image-quality determination. In yet a further preferred embodiment 
of this method, an artificial neural network processor trained to recognize the visual 
images characterized by an image-quality level that fall outside the predetermined image- 
quality range is utilized. 



Features and Advantages of the Present Invention 

It is a feature of the present invention that photographic prints are 
inspected, and defective prints are rejected without human intervention. 

It is another feature of the present invention that the results of the print 
inspection are consistent 

It is a further feature of the present invention that the print inspection 
process can be accomplished at speeds up to 5 times faster than the typical method 
involving human interaction. 

An advantage of the present invention is that it lowers the cost of 
inspecting photographic prints because no human intervention is required. 

A further advantage of the present invention is that it speeds up the print 
inspection process because, unlike human-based systems, the system of the present 
invention operates continuously and at speeds attainable only through the use of 
machines and computer programs. 

Additional features and advantages of the invention are set forth in part ir 
the description that follows, and in part are apparent from the description, or may be 
learned by practice of the invention. The features and advantages of the invention may 
also be realized and attained by means of the instrumentalities and combinations 
particularly set out in the appended claims. 
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BRIEF DESCRIPTION OF THE FIGURES 

The accompanying drawings, which are incorporated in and constitute part 
of the specification, illustrate preferred embodiments of the invention, and, together with 
the description, serve to explain the principles of the present invention. In the drawings, 
5\ like reference numbers indicate identical or functionally similar elements. Additionally, 
the left-most digit(s) of a reference number identifies the drawing in which the reference 

number first appears. 

FIG. 1 shows a block diagram of one embodiment of a print inspection 

system according to the present invention, 
j 0 FIG. 2 depicts a flow diagram of the training phase of an artificial neural 

network processor trained to distinguish between good prints and reject prints. 

FIGs. 3 A and 3B depict a flow diagram for a method for inspecting prints 
in accordance with the present invention. 

FIG. 4 depicts an exemplary computer system, suitable for use with the 

1 5 present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Reference will now be made in detail to the preferred embodiments of the 
invention, examples of which are illustrated in the accompanying drawings. Notably, the 

20 present invention may be implemented using software, hardware or any combination 
thereof, as would be apparent to those of ordinary skill in the art, and the figures and 
examples below are not meant to limit the scope of the present invention or its 
embodiments or equivalents. 

Overview of the Present Invention 

25 The present invention makes it possible for new and existing photo 

processing systems to efficiently and accurately perform an inspection of prints and reject 
those prints that are defective, all without human intervention. An electronic processor is 
equipped with an artificial neural network that is trained to accept or reject prints based 
on histograms of the image data, statistical abstractions, such as minimum, maximum 

30 and standard deviation, range, etc., on raw pixel data or a subset of raw pixel data 

(sampled data). The prints are tracked as they move through the system on a continuous 
web of paper, which makes it possible to detect and sort out the defective prints. 

-5- 
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Detailed Operation f the Present Invention 

With reference now to FIG. 1, a block diagram of one embodiment of a 
print inspection system 104 in accordance with the present invention is shown. The print 
inspection system is comprised of a transport mechanism, shown as rollers 108, 109 and 
1 1 0 in FIG. 1, a scanner 124, a digitizer 128 and an electronic processor 136. Transport 
mechanism rollers 108, 109 and 1 10 move a length of paper 1 12 along a work path 1 16. 
The length of paper 1 12 contains a plurality of prints 120, 121, 122 and 123 to be 
processed by print-processing devices disposed along the length of the work path 116. 

The scanner 124 captures a visual image of each print and sends analog 
signals representing the visual image to digitizer 128 via link 126. In the preferred 
embodiment, digitizer 128 comprises an analog-to-digital converter (not shown) suitable 
for converting the analog signals from scanner 124 into a digital file corresponding to the 
visual image contained on each print 120, 121, 122 and 123. In practice, digitizer 128 
may reside in electronic processor 136, as shown in FIG. 1, or it could reside in scanner 
1 24. an alternate embodiment not illustrated for brevity. Once the digital file is created, it 
is sent via link 134 to an artificial neural network processor 148, which determines, based 
on operations performed on the digital file, whether an image-quality level that fells 
outside a predetermined image-quality range characterizes the visual image. A person of 
skill in the art would recognize that the system could alternatively be implemented in 
such a way that it determines whether the image-quality level falls inside a predetermined 
image-quality range, instead of outside of it. A detailed description of the training and 
operation of the artificial neural network 148 is provided below. 

In the preferred embodiment, the print inspection system of the present 
invention includes a storage medium 140, coupled to digitizer 128 via link 130 and the 
artificial neural network processor via link 1 42, for storing the digital files corresponding 
to the visual images contained on prints 120, 121, 122 and 123. Storage medium 140 
could comprise any mass storage device, such as a hard drive, CDROM drive, DVD 
drive, tape drive, etc. In the most preferred embodiment, print inspection station 104 also 
includes a second storage medium 144, linked to artificial neural network processor 148 
through link 146, for storing the results of the analysis carried out by artificial neural 
network 148. In practice, second storage medium 144, which could also be comprised of 
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any mass storage device, may be a different device or the same device as storage medium 
140. 

In a further embodiment of the present invention, print inspection station 
1 04 further comprises a sort marker 1 52. When artificial neural network processor 148 
5 determines that a visual image is characterized by an image-quality level outside a 

predetermined range, it sends a notification to other print processing devices, such as sort 
marker 152, indicating that the print corresponding to that visual image should be 
rejected. In this case, the notification signal is sent to sort marker 152 via link 150. In 
response to the notification, sort marker 152 marks the print in order to distinguish it 

1 0 from prints that are not to be rejected. For example, in the preferred embodiment, sort 
marker 152 applies an infrared absorbent sticker or ink-jet printer mark to the defective 
print. Other devices disposed along the work path 116, such as a packaging station 156, 
detect the sticker or ink-jet printer mark via an infrared reader, for example, and removes 
the defective print from the batch. In the embodiment depicted in FIG. 1, packaging 

1 5 station 1 56 cuts the paper 112 and sorts defective prints 1 80 into reject bin 1 88 anil good 
prints 1 84 into keep bin 1 90. 

In an alternative embodiment, the reject notification is sent from electronic 
controller 136 to a batch manager 160 via link 159. The batch manager 160 maintains an 
electronic record of each print's location in work path 116 and each print's image-quality 

20 status. The electronic records are updated continuously as prints move along work path 
116. To facilitate the tracking of prints in work path 1 1 6, a cut marker 1 64 is positioned 
near the beginning of work path 116. Typically, cut markers are located inside the 
photographic image printer from which the prints came. 

Cut marker 164 identifies the leading edge of each print, the trailing edge 

25 of each print, or both, as they enter the work path 116, and inserts cut marks 

corresponding to one or both edges on the length of paper 112. The cut marks are 
detected by cut mark sensors 168, 169, 170 and 171, which send location information for 
each print to electronic controller 136 via links 172, 173, 174 and 175, respectively. It is 
understood that cut mark sensors 168, 169, 170 and 171 could also be directly coupled to 

30 batch manager 160, scanner 124, or any other print processing device, as well as 

electronic controller 136. Indeed, in the most preferred embodiment of the present 
invention, the cut mark sensors trigger the image capture performed by scanner 124. It is 
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further understood that electronic controller 136, or a process or program residing within 
electronic controller 136, could implement the scan, cut mark sensing or batch manager 
functions. 

Since batch manager 1 60 maintains a continuously updated electronic 
5 record of each print's location and each print's image-quality status, the print inspection 
station 104 of this embodiment would not require sort marker 152 or any infrared light 
absorbent stickers or ink-jet printer marks to identify defective prints. Instead, batch 
manager 160 transmits the appropriate accept or reject command to packaging station 156 
at the appropriate time via link 158. In a preferred embodiment, batch manager 1 60 also 
10 sends other print-related data, such as size of the order, image size, envelope number, 

paper type, etc., to packaging station 156 or other print processing devices. In yet another 
alternative embodiment, the accept/reject notification is sent from electronic controller 
1 36 directly to a packaging station 1 56 via link 1 54. Finally, packaging station 1 56 sorts 
the defective print 180 into discard bin 188 and the good print 184 into keep bin 190. 
1 5 As would be apparent to one skilled in the art, the process of using an 

artificial neural network ("ANN") to solve problems involves two distinct phases: the 
training phase and operational phase. The training phase comprises the activities of: (1) 
building a 'training set" for the ANN based on a representative sample and "correct" 
results as defined by a human trainer, and (2) repeatedly exposing the training set 
20 samples to the ANN along with the correct results for each sample until the ANN has 
"learned" how to derive the correct result for each sample on its own. In reality, the 
ANN does not actually "learn" how to derive the correct result, but generates an internal 
set of mathematical rules, which, when applied to the sample inputs, yields substantially 
the same results reached by the human trainer for each of the sample inputs. 
25 At the end of the training phase, the ANN is considered to be "trained," 

which means it should be capable of solving similar, but new problems. The operational 
phase comprises exposing the "trained" ANN to new images that are similar, but not 
identical to those in the training set, and allowing the neural network to "decide" whether 
the new images meet the criteria defined by its internal rules. 
30 There are a number of general purpose ANNs, suitable for purposes of 

inspecting prints in accordance with the present invention commercially available. The 
ANN known as Trajan 4.0, manufactured and sold by Trajan Software Ltd., for example, 

-8- 



WO 01/14928 PCT/USOO/22352 

more than adequately performs the job described herein^ It will be recognized by those 
with skill in the art that any multi-layer perceptron ANN with suitable back-propagation 
training software would be suitable for these purposes. 

In the preferred embodiment of the present invention, an ANN is trained to 
5 distinguish good prints from reject prints an abstraction of the image data, such as a set of 
histograms. In another preferred embodiment, the ANN recognizes good and bad prints 
based on raw pixel data, or a subset (or sample) of raw pixel data for each print image. In 
another embodiment, the ANN recognizes good prints and bad prints based on certain 
statistics computed from the image data, such as the maximum, minimum, average and 

10 standard deviation values of raw pixels. In these cases, specialized analog hardware, 
such as video peak detection and video signal averaging, can be utilized to provide 
certain of these statistical values in video real time for greatly enhanced accuracy. In 
another preferred embodiment, the ANN is trained to operate based on raw pixel data or a 
subset (or sample) of the raw pixel data. 

15 A more detailed description of how to train and operate an artificial neural 

network in accordance with one embodiment of the present invention is now provided. In 
the first example, the ANN is trained to distinguish between good prints and reject prints 
based on a histogram of the image data. A histogram is a data array which consists of a 
number of variables each of which represents a range of pixel values in the sampled 

20 image. The value of each variable is equal to the number of pixels in the sampled image 
with values falling within the range of pixel values assigned to that variable. In the 
preferred embodiment, a histogram is used for input to the ANN because the histogram 
accounts for all pixels in its value range, even if pixels within the value range represent 
only a very small portion of the overall image. 

25 With reference again to the flow diagram 200 depicted in FIG. 2, the first 

step for training the ANN to distinguish good prints from reject prints, is to decide how 
many points (pixels) will be sampled from each visual image and how many inputs and 
outputs are required for the ANN. See step 202 in FIG. 2. In a preferred embodiment, 
307,200 points are used, since it provides a reasonably accurate representation of the 

30 quality level of a 640 x 480 size image, which is the standard resolution for a video 
graphics array ("VGA") monitor. 

-9- 
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The next step, step 204, is to decide how many inputs (also referred to 
herein as the number of "cells" or "nodes") and outputs the ANN will have. In the 
preferred embodiment, the number of inputs to the ANN is defined as the square root of 
the number of points sampled from each visual image. Thus, when the sample size is 
307,200 points, it works well to define an ANN with 554 inputs. In a preferred 
embodiment, a histogram is created for each cell. It is to be understood, however, that 
the present invention is not limited to the foregoing number of sample points, neural 
network inputs and histograms; alternative numbers for image sample sizes, inputs and 
histograms may be used without departing from the scope of the invention. The number 
of outputs used for the preferred embodiment of the ANN is 1, which corresponds to the 
accept or reject decision. 

After the image sample size, number of input cells (or nodes) and outputs 
in the ANN and histograms have been determined, the next step, shown as step 206 in 
FIG. 2, is to pick a representative population of real world prints to be used in the ANN 
testing set. In a preferred embodiment, an adequate rule of thumb is to use approximately 
1 00 prints for each input node in the ANN. In this case, 5,540 prints would be used. It is 
understood, however, that the present invention is not limited to any particular number of 
prints used in the testing set 

The population of prints selected for the testing set should include a 
reasonable number of both "good" and "bad" prints. The "good" prints should be 
reasonably consistent, in tenns of subject matter and image-quality, with the types of 
photographs for which consumers are willing to pay and photo processors are willing to 
charge. The "bad" prints should consist of prints having the most common types of 
defects that render photographs unmarketable, such as photographs that are blank, 
fogged, torn, over- or under-exposed, etc. 

The next step, depicted in flow diagram 200 as step 208, is to capture the 
visual image of each print in the population of test prints, preferably with scanner 124, 
and convert each visual image to a digital file corresponding to the visual image. In a 
step 2 1 0, histograms are created for each digital file. Then, in a step 212, the digital files 
are displayed to a human being, who makes a subjective determination as to whether the 
image should be "accepted" or "rejected." The effect of this determination is to tell the 
ANN what the human being considers to be the "correct" result. Next, in a step 214, the 

- 10- 



WO 01/14928 PCT/USO0/22352 
histogram data, along with the human being's accept or reject decision, are assigned to a 

testing file, also known as a "training file." 

Then, the final step, step 21 6, in the training phase is to repeatedly execute 

the ANN on the training set until the ANN is capable of reaching the correct conclusion 

5 V . (accept or reject) on all the images in the sample set on its own. At this point the training 
of the ANN is complete and the ANN can be utilized in the print inspection system 104, 
as described above with reference to FIG. 1, to render accept/reject decisions not on 
sample or test prints, but real prints. 

With reference again to the flow diagram 200 in FIG. 2, the training steps 

1 0 for an ANN which distinguishes good prints and bad prints based on a subset of raw pixel 
data are now described. The process is very similar to the process for training an ANN to 
distinguish based on image histogram data. First, a determination is made as to the 
number of points (pixels) that will be sampled from each visual image. See step 202. As 
in the histogram example above, 307,200 points are used in the preferred embodiment, 

1 5 since it provides a reasonably accurate representation of the quality level of the entire 
image. 

Next, the number of input nodes (cells) and output nodes for the ANN is 
decided. See step 204. Again, the square root of the number of sample points is a good 
rule-of-thumb for determining the number of input nodes. Thus, the ANN will have 554 

20 input nodes (cells). Again, as in the histogram example, it is to be understood that the 
present invention is not limited to the foregoing number of sample points and neural 
network inputs; alternative image sample sizes and ANN input sizes may be used without 
departing from the scope of the invention. The number of outputs for the ANN is 1, 
which corresponds to the accept or reject decision. 

25 After the image sample size and the number of nodes in the ANN have 

been determined, the next step, shown as step 206 in FIG. 2, is to pick a representative 
population of real world prints to be used in the ANN testing set. In a preferred 
embodiment, an adequate rule of thumb is to use approximately 100 prints for each input, 
or 5,540 prints. It is also understood, however, that the present invention is not limited 

30 to any particular number of prints used in the testing set. 

The next step, depicted in flow diagram 200 as step 208, is to capture the 
visual image of each print in the population of test prints, preferably with scanner 124, 

- 11 - 
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and convert each visual image to a digital file corresponding to the visual image. In this 
case, no histogram is created from the digital files, so step 210 would be skipped. Next, 
in a step 212, the digital files are displayed to a human being, who makes a subjective 
determination as to whether the image should be "accepted" or "rejected," thereby telling 
5 the ANN what the human being considers to be the "correct" result. Next, in a step 214, 
the raw pixel data from the digital files, along with the human being's accept or reject 
decisions, are assigned to the "training file." 

Then, the final step in the training phase, step 216 in the diagram, is to 
repeatedly execute the ANN on the training set until the ANN is capable of reaching the 
10 correct conclusion (accept or reject) on all the images in the sample set on its own. Now 
the training of the ANN is complete and the ANN can be utilized in the print inspection 
system 104, described above with reference to FIG. 1, to render accept/reject decisions on 
real prints. 

Training an ANN to recognize defects in visual images based on statistical 

1 5 abstractions is accomplished the same way training was accomplished for histogram data 
and raw pixel data in the previous two examples, except that maximum, minim u m , 
average and standard deviation values for the sample points are used as the inputs to the 
ANN instead of histograms or raw pixel data. 

With reference to the flow diagram depicted in FIGs. 3 A and 3B, a more 

20 complete description of a method for inspecting prints according to the present invention 
is provided. In a step 302, a length of paper 1 12 containing a plurality of prints 120-124 
is fed into a work path 116. In the photo processing industry, this length of paper 1 12 is 
known as a "continuous web." In a step 304, a visual image of each print in the length of 
paper 1 12 is captured, typically by a scanner 124. Scanner 124 may be implemented 

25 using a color sequential exposure system with a monochrome camera to lower costs, but a 
color video camera would also work well. The visual image for each print is then 
converted into a corresponding digital file in a step 306. 

In a step 308, the digital file corresponding to the visual image (and the 
print) to be inspected is selected. Then, using the artificial neural network processor 148 

30 described above, for example, a determination is made in step 3 10 as to whether the 

visual image corresponding to the digital file is characterized by an image-quality level 
outside a predetermined range. If the answer is "NO," then processing loops back to step 
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308, where another digital file corresponding to another visual image (and another print) 
is selected for processing. It is recognized that, in alternative embodiments, certain 
actions could be taken upon a determination that there are no defects in a print, such as 

marking the print "good." 

If the visual image is determined to be characterized by an image-quality 
level outside a predetermined range (the answer is "YES"), then processing continues in 
the flow diagram shown on FIG. 3B by way of flow connector 300B. As shown in FIG. 
3B at step 3 1 2, a notification of a defective print is sent to other print processing devices 
in the print inspection system 104. Some of these devices, such as the previously- 
described sort marker 152 or packaging station 156, could be disposed along the work 
path 1 16. Other devices, such as batch manager 160 or second storage medium 144, may 
not be disposed on the work path 1 16, but receive the information for subsequent 
processing. The notification may also be sent to still other print processing devices (not 
shown), such as a website server connected to an interconnected computer network like 
the Internet, so that a consumer or a retail clerk can log on from a remote location to 
check the image-quality status of a particular set of prints. 

If the notification is sent to a sort marker 152, then the sort marker 152 
applies an infrared light absorbent sticker or ink-jet printer mark to the defective print in a 
step 3.14. If the notification is sent to a batch manager 160, then, in a step 316, batch 
manager lbO updates i t s elec t ronic l ec ord fui diat paiticular printrand,^at the apprepnate- 
time, sends a reject instruction to packaging station 156. Finally, in a step 318, packaging 
station 1 56 sorts the defective print 1 80 into reject bin 1 88 and the good prints 1 84 into 
keep bin 190. The defective prints are then ready for discarding or further processing, as 
the case may be. Step 320 determines whether there are more prints to be inspected. If 
the answer is "YES," then processing returns to step 308 by way of flowchart connector 
30A. Otherwise, processing is complete. 

With reference now to FIG. 4, a more complete description of a computer system suitable 
for use with the preferred embodiment of the present invention is provided. The 
computer system 402 includes one or more processors, such as a processor 404. The 
30 processor 404 is connected to a communication bus 406. Various software embodiments 
described in terms of this exemplary computer system. After reading this description, 
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it will become apparent to a person skilled in the relevant art how to implement the 
invention using other computer systems and/or computer architectures. 

The computer system 402 also includes a main memory 408, preferably random 
access memory (RAM), and can also include a secondary memory 410. The secondary 
\5 memory 410 can include, for example, a hard disk drive 412 and/or a removable storage 
drive 414, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, 
etc. The removable storage drive 414 reads from and/or writes to a removable storage 
unit 41 8 in a well-known manner. The removable storage unit 418, represents a floppy 
disk, magnetic tape, optical disk, etc. which is read by and written to by the removable 

1 0 storage drive 414. As will be appreciated, the removable storage unit 418 includes a 
computer usable storage medium having stored therein computer software and/or data. 

In alternative embodiments, the secondary memory 410 may include other similar 
means for allowing computer programs or other instructions to be loaded into the 
computer system 402. Such means can include, for example, a removable storage unit 

1 5 422 and an interface 420. Examples of such can include a program cartridge and cartridge 
interface (such as that found in video game devices), a removable memory chip (such as 
an EPROM, or PROM) and associated socket, and other removable storage units 422 and 
interfaces 420 which allow software and data to be transferred from the removable 
storage unit 422 to the computer system 402. 

20 The computer system 402 can also include a communications interface 424. The 

communications interface 424 allows software and data to be transferred between the 
computer system 402 and external devices. Examples of the communications interface 
424 can include a modem, a network interface (such as an Ethernet card), a 
communications port, a PCMCIA slot and card, etc. Software and data transferred via the 

25 communications interface 424 are in the form of signals 426 that can be electronic, 
electromagnetic, optical or other signals capable of being received by the 
communications interface 424. Signals 426 are provided to communications interface via 
a channel 428. A channel 428 carries signals 426 and can be implemented using wire or 
cable, fiber optics, a phone line, a cellular phone link, an RF link and other 

30 communications channels. 

In this document, the terms "computer program medium" and "computer usable 
medium" are used to generally refer to media such as the removable storage device 418, a 



- 14- 



WO 01/14928 PCT/US00/22352 
hard disk installed in hard disk drive 412, and signals 426. These computer program 
products are means for providing software to the computer system 402. 

Computer programs (also called computer control logic) are stored in the main 
memory 408 and/or the secondary memory 410. Computer programs can also be 
5 received via the communications interface 424. Such computer programs, when 
executed, enable the computer system 402 to perform the features of the present 
invention as discussed herein. In particular, the computer programs, when executed, 
enable the processor 404 to perform the features of the present invention. Accordingly, 
such computer programs represent controllers of the computer system 402. 

10 In an embodiment where the invention is implemented using software, the 

software may be stored in a computer program product and loaded into the computer 
system 402 using the removable storage drive 414, the hard drive 412 or the 
communications interface 424. The control logic (software), when executed by the 
processor 404, causes the processor 404 to perform the functions of the invention as 

15 described herein. 

In another embodiment, the invention is implemented primarily in hardware 
using, for example, hardware components such as application specific integrated circuits 
(ASICs). Implementation of such a hardware state machine so as to perform the 
functions described herein will be apparent to persons skilled in the relevant art(s). 

20 In yet another embodiment, the invention is implemented using a combination of 

both hardware and software. 

The present invention has been disclosed and described herein in what is 
considered to be its most preferred embodiments. It should be noted that variations and 
equivalents may occur to those skilled in the art upon reading the present disclosure and 

25 that such variations and equivalents are intended to come within the scope of the 
invention and the appended claims. 
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CLAIMS 
What I claim is: 

1 . A print inspection system, comprising: 

a transport mechanism for moving a length of paper along a work path, 
said length of paper containing a plurality of prints; 

a scanner positioned along said work path, said scanner being configured 

to capture a visual image of each of said plurality of prints; 

j> 

means, coupled to said scanner, for converting said visual image into a 
digital file corresponding to said visual image; and 

an electronic processor, wherein said electronic processor determines 
whether said visual image is characterized by an image-quality 
level that falls outside a predetermined image-quality range. 

The print inspection system of claim 1 , wherein said electronic processor is 
comprised of: 

a neural network processor that is trained to recognize a digital file having 
an image-quality level that falls outside a predetermined range of 
image-quality levels. 



The print inspection system of claim 1 , further comprising means for storing said 
digital file on a first storage medium. 
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4. 



The print inspection system of claim 3, further comprising means for storing the 
result of said image-quality level determination on a second storage 



medium. 



5 



5. 



The print inspection system of claim 1, further comprising: 



means, responsive to said electronic processor, for marking prints 

corresponding to visual images characterized by an image-quality 
level that falls outside said predetermined image-quality range. 

The print inspection system of claim 5, further comprising: 

means for discarding prints corresponding to visual images having image- 
quality levels that fall outside said predetermined image-quality 
range. 

The print inspection system of claim 1 , further comprising: 

means for tracking the current location of a print as it moves along said 
work path; and 

means for transmitting the image-quality level of said print and current 
location of that print on said work path to a print-processing 



The print inspection system of claim 1, further comprising means for identifying 
leading and a trailing edge of each print as it moves along said work path. 



20 



device. 



- 17- 



n- <Wn OU4q?8A2 I > 



WO 01/14928 



PCT/US00/22352 



The print inspection system of claim 8, wherein said means for identifying 
comprises: 

a cut-marker positioned near the start of said work path, wherein said cut- 
marker inserts cut-marks on said length of paper corresponding to the 
trailing edge of each print; and 

a cut-mark sensor. 



1 0. The print inspection system of claim 7, wherein said means for tracking 
comprises: 

a cut-marker positioned near the start of said work path, wherein said cut- 
marker inserts cut-marks on said length of paper corresponding to 
the trailing edge of each print; and 

a cut-mark sensor. 

II. A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 
capturing a visual image of each print; 

converting said visual image of each print into a digital file corresponding 
to said visual image; 

determining whether said visual image is characterized by an image- 
quality level that falls outside a predetermined image-quality 
range; and 

repeating said determining step for each digital file created in said 
converting step. 
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1 2. The method of claim 1 1 , wherein said determining step is carried out using an 
electronic processor. 



The method of claim 11, wherein said determining step comprises: 

training a neural network processor to recognize visual images 

characterized by an image-quality level that falls outside said 
predetermined image-quality range. 



14. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that contains no 
useful content. 

1 5. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that has a 
physical defect. 

1 6. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that has a 
repetitive exposure error. 

17. The method of claim 13, wherein said training step is accomplished by presenting 
said neural network processor with a histogram of the sampled visual image. 
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1 8. The method of claim 13, wherein said training is accomplished by presenting said 
neural network processor with statistical abstractions of sample visual 
images. 



1 9. The method of claim 13, wherein said training comprises presenting said neural 
network processor with raw pixel data corresponding to sample visual 
images. 



20. The method of claim 13, wherein said training is accomplished by presenting said* 
neural network processor with a subset of raw pixel data corresponding to 
sample visual images. 



21 . The method of claim 1 1, further comprising the step of storing said digital file on 
a first storage medium. 



22. The method of claim 1 1, further comprising the steps of: 

marking prints corresponding to visual images characterized by an image- 
quality level that falls outside said predetermined image-quality 
range to form marked prints; and 

discarding said marked prints containing said markings. 



23. The method of claim 1 1, further comprising the step of tracking the current 
location of each print as it moves along said work path. 
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24. The method of claim 23, further comprising the step of: 

transmitting image-quality level of a print and current location of that print 
on said work path to a print-processing device. 



25. 



The method of claim 24, wherein said image-quality level and current location are 
transmitted to said print-processing device via a link to an interconnected 
j o computer network. 

26. A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 

marking a leading or trailing edge of each print with cut-marks; 

sensing the location of said cut-marks as said length of paper moves along 
j ^ said work path; and 

in response to said sensing, 

capturing a visual image of each print, 

converting said visual image into a digital file corresponding to 
said visual image, and 



deterniining whether said visual image is characterized by an 
image-quality level that falls outside a predetermined 
image-quality range. 



27. The method of claim 26, wherein said determining step is carried out using an 
25 electronic processor. 
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The method of claim 26, wherein said determining step comprises: 

training a neural network processor to recognize visual images 

characterized by an image-quality level that falls outside said 
predetermined image-quality range. 

A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 
capturing a visual image of each print; 

converting said visual image of each print into a digital file corresponding 
to said visual image; 

determining via an electronic processor whether said visual image is 
characterized by an image-quality level that falls outside a 
predetermined image-quality range; and 

repeating said determining step for each digital file created in said 
converting step. 
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RELATED APPLICATIONS 

This application claims priority under 35 U.S.C. 1 1 9 to provisional 
application 60/150,075, filed August 20, 1999, the entirety of which is incorporated 
herein by reference. 

FIELD OF THE INVENTION 

The present invention relates generally to the field of photography, and 
more particularly to photographic film processing. Still more particularly, the present 
invention relates to the task of inspecting photographic prints for defects in exposure a 
image-quality and sorting out defective prints before they are sent to the customer. 



RELATED ART 

Photographic prints are typically inspected in photo finishing laboratories 
by trained operators who manually apply infrared absorbent stickers to prints that come 
out of the developing process blank, fogged or otherwise damaged beyond a level at 
which the print can be sold. In addition to problems caused by over and under exposure, 
photographic prints can have physical defects, such as scratches, water spots, edge 
damage or emulsion tears. Sometimes prints are damaged by repetitive exposure errors. 
These kinds of defects, which can be the result of a hair or film chip falling onto the 
exposure mechanism of the image printer, for example, can affect a large number of 
prints. 

Current methods for inspecting prints require human intervention. A 
human being must manually examine every single print to determine whether the print 
should be rejected. Unfortunately, photo processing is much more expensive and time- 
consuming when a human being is involved. For example, when a human inspects prints 
moving on an assembly line, the human must periodically stop the assembly line in order 
to afford the time necessary to closely examine a particular print and make a decision 
about whether the print should be accepted or rejected. Thus, the whole photo processing 
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job is slowed down tremendously because of the human interaction, which causes the 
process to stop and start over and over again. 

Many human operators are trained in the art of recognizing defects. But 
the training itself is expensive and time consuming. Plus, an employer never knows 
whether he will see a return on training investment dollars because human beings tend to 
quit, retire, take other jobs, get sick, or otherwise stop performing the inspection task for 
which they have been trained. Moreover, no matter how well a human is trained in the 
art of inspecting prints, the decision to accept or reject a particular print is ultimately an 
entirely subjective decision. Thus, even for a group of highly trained individuals, results 
will be inconsistent, depending on which operator is doing the inspection. 

Another way of overcoming the problems with bad prints is to use very 
sophisticated printing equipment Typically, these printers operate by scanning the image 
contained in a frame of film in a number of different ways to ascertain which bands of 
light are present as a result of incorrect exposure. These printers then either attempt to 
print the image in such a way as to overcome the defect, or simply skip images that 
contain defects printing adjustments cannot overcome. Unfortunately, there are some 
very significant drawbacks to using printers to identify and reject defective prints. 

First, the printers are extremely expensive. Second, they add an 
unacceptable amount of additional time to the photo finishing process. Third, they do not 
address defects that come about as a result of something that occurs during or after the 
image printing process, such as foreign objects falling in the optical path, physical paper 
damage, defective printing equipment, incorrect paper development and numerous other 
problems, some of which are not immediately obvious. And fourth, using more 
sophisticated printers has not been found to be as reliable for rejecting defective prints as 
using human beings, mainly because printers can only identify incorrect exposures within 
a very limited range. 

Accordingly, there is a need recognized by inventors in the photo 
processing industry for fast, consistent and economical method for detecting and rejecting 
prints that are defective without human intervention. 
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SUMMARY OF THE INVENTION 

The present invention is directed to an automated system and method for 
inspecting photographic prints, detecting defects in those prints, and removing the 
5, defective prints from the job. In one aspect of the present invention, a print inspection 
system is provided that comprises: a transport mechanism for moving a length of paper 
containing a plurality of prints along a work path; a scanner, positioned along the work 
path, configured to capture a visual image of each print; means, coupled to the scanner, 
for converting the visual image of each print into a digital file corresponding to the visual 
10 image; and an electronic processor that determines whether the visual image is 

characterized by an image-quality level that falls inside a predetermined image-quality 
range. In the preferred embodiment, the electronic processor is comprised of an artificial 
neural network processor trained to recognize digital files having image-quality levels 
that fall outside the predetermined range. 

The print inspection station of the present invention can also comprise 
for storing the digital files corresponding to the visual images contained on the 
prints in a storage medium for later processing. In a further preferred embodiment, the 
result of the image-quality level determination can also be stored in a storage medium. 

Although the above-summarized embodiments are directed to detecting 
20 prints characterized by an image-quality level that falls outside a predetermined range, 
person of skill in the art would recognize that the same results could be achieved by 
alternate means, such as detecting prints characterized by image-quality levels that fall 
inside a predetermined range. Such alternate means are considered within the scope of 

the present invention. 

In another aspect of the present invention, prints determined to image 
quality levels that fall outside a predetermined range are marked as defective. In yet 
another aspect of the present invention, prints so marked are automatically discarded. In 
still another aspect of the current invention, the print inspection system also includes 
means for tracking the current location of a print as it moves along the work path. This 
preferred embodiment also includes means for transmitting the image-quality level and 
current location of that print within the work path to another print processing device. 
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In still a further aspect of the present invention, a method or process for inspecting prints 

is provided. The method comprises the steps of: feeding into a work path a length of 
paper containing a plurality of prints; capturing a visual image of each print; converting 
the visual image of each print into a digital file corresponding to the visual image; 
determining whether the visual image is characterized by an image-quality level that falls 
outside a predetermined image-quality range; and repeating the determining step for each 
digital file created. In the preferred embodiment of this method, an electronic processor 
is used to make the image-quality determination. In yet a further preferred embodiment 
of this method, an artificial neural network processor trained to recognize the visual 
images characterized by an image-quality level that fall outside the predetermined image- 
quality range is utilized. 

Features and Advantages of the Present Invention 

It is a feature of the present invention that photographic prints are 
inspected, and defective prints are rejected without human intervention. 

It is another feature of the present invention that the results of the print 
inspection are consistent 

It is a further feature of the present invention that the print inspection 
process can be accomplished at speeds up to 5 times faster than the typical method 
involving human interaction. 

An advantage of the present invention is that it lowers the cost of 
inspecting photographic prints because no human intervention is required. 

A further advantage of the present invention is that it speeds up the print 
inspection process because, unlike human-based systems, the system of the present 
invention operates continuously and at speeds attainable only through the use of 
machines and computer programs. 

Additional features and advantages of the invention are set forth in part in 
the description that follows, and in part are apparent from the description, or may be 
learned by practice of the invention. The features and advantages of the invention may 
also be realized and attained by means of the instrumentalities and combinations 
particularly set out in the appended claims. 
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BRIEF DESCRIPTION OF THE FIGURES 

The accompanying drawings, which are incorporated in and constitute part 
of the specification, illustrate preferred embodiments of the invention, and, together with 
the description, serve to explain the principles of the present invention. In the drawings, 
5 - like reference numbers indicate identical or functionally similar elements. Additionally, 
the left-most digit(s) of a reference number identifies the drawing in which the reference 
number first appears. 

FIG. 1 shows a block diagram of one embodiment of a print inspection 
system according to the present invention. 
I o FIG. 2 depicts a flow diagram of the training phase of an artificial neural 

network processor trained to distinguish between good prints and reject prints. 

FIGs. 3 A and 3B depict a flow diagram for a method for inspecting prints 
in accordance with the present invention. 

FIG. 4 depicts an exemplary computer system, suitable for use with the 
1 5 present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Reference will now be made in detail to the preferred embodiments of the 
invention, examples of which are illustrated in the accompanying drawings. Notably, the 

20 present invention may be implemented using software, hardware or any combination 
thereof, as would be apparent to those of ordinary skill in the art, and the figures and 
examples below are not meant to limit the scope of the present invention or its 
embodiments or equivalents. 

Overview of the Present Invention 

25 The present invention makes it possible for new and existing photo 

processing systems to efficiently and accurately perform an inspection of prints and reject 
those prints that are defective, all without human intervention. An electronic processor is 
equipped with an artificial neural network that is trained to accept or reject prints based 
on histograms of the image data, statistical abstractions, such as minimum, maximum 

30 and standard deviation, range, etc., on raw pixel data or a subset of raw pixel data 

(sampled data). The prints are tracked as they move through the system on a continuous 
web of paper, which makes it possible to detect and sort out the defective prints. 
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Detailed Operation of the Present Inventi n 

With reference now to FIG. 1, a block diagram of one embodiment of a 
print inspection system 104 in accordance with the present invention is shown. The print 
inspection system is comprised of a transport mechanism, shown as rollers 108, 109 and 
S 1 10 in FIG. 1, a scanner 124, a digitizer 128 and an electronic processor 136. Transport 
mechanism rollers 108, 109 and 110 move a length of paper 112 along a work path 116. 
The length of paper 112 contains a plurality of prints 120, 12 1, 122 and 123 to be 
processed by print-processing devices disposed along the length of the work path 1 16. 

The scanner 124 captures a visual image of each print and sends analog 

10 signals representing the visual image to digitizer 128 via link 126. In the preferred 

embodiment, digitizer 128 comprises an analog-to-digital converter (not shown) suitable 
for converting the analog signals from scanner 124 into a digital file corresponding to the 
visual image contained on each print 120, 121, 122 and 123. In practice, digitizer 128 
may reside in electronic processor 136, as shown in FIG. 1, or it could reside in scanner 

1 5 1 24, an alternate embodiment not illustrated for brevity. Once the digital file is created, it 
is sent via link 134 to an artificial neural network processor 148, which determines, based 
on operations performed on the digital file, whether an image-quality level that falls 
outside a predetermined image-quality range characterizes the visual image. A person of 
skill in the art would recognize that the system could alternatively be implemented in 

20 such a way that it determines whether the image-quality level falls inside a predetermined 
image-quality range, instead of outside of it. A detailed description of the training and 
operation of the artificial neural network 148 is provided below. 

In the preferred embodiment, the print inspection system of the present 
invention includes a storage medium 140, coupled to digitizer 128 via link 130 and the 

25 artificial neural network processor via link 142, for storing the digital files corresponding 
to the visual images contained on prints 120, 121, 122 and 123. Storage medium 140 
could comprise any mass storage device, such as a hard drive, CDROM drive, DVD 
drive, tape drive, etc. In the most preferred embodiment, print inspection station 104 also 
includes a second storage medium 144, linked to artificial neural network processor 148 

30 through link 146, for storing the results of the analysis carried out by artificial neural 

network 148. In practice, second storage medium 144, which could also be comprised of 
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any mass storage device, may be a different device or the same device as storage medium 
140. 

In a further embodiment of the present invention, print inspection station 
1 04 further comprises a sort marker 152. When artificial neural network processor 148 
5/ determines that a visual image is characterized by an image-quality level outside a 

predetermined range, it sends a notification to other print processing devices, such as sort 
marker 1 52, indicating that the print corresponding to that visual image should be 
rejected. In this case, the notification signal is sent to sort marker 152 via link 150. In 
response to the notification, sort marker 152 marks the print in order to distinguish it 

1 0 from prints that are not to be rejected. For example, in the preferred embodiment, sort 
marker 1 52 applies an infrared absorbent sticker or ink-jet printer mark to the defective 
print. Other devices disposed along the work path 116, such as a packaging station 156, 
detect the sticker or ink-jet printer mark via an infrared reader, for example, and removes 
the defective print from the batch. In the embodiment depicted in FIG. 1, packaging 

1 5 station 1 56 cuts the paper 1 12 and sorts defective prints 1 80 into reject bin 1 88 and good 
prints 1 84 into keep bin 190. 

In an alternative embodiment, the reject notification is sent from electronic 
controller 1 36 to a batch manager 160 via link 159. The batch manager 160 maintains an 
electronic record of each print's location in work path 116 and each print's image-quality 

20 status. The electronic records are updated continuously as prints move along work path 
116. To facilitate the tracking of prints in work path 1 1 6, a cut marker 1 64 is positioned 
near the beginning of work path 116. Typically, cut markers are located inside the 
photographic image printer from which the prints came. 

Cut marker 164 identifies the leading edge of each print, the trailing edge 

25 of each print* or both, as they enter the work path 116, and inserts cut marks 

corresponding to one or both edges on the length of paper 1 12. The cut marks are 
detected by cut mark sensors 168, 169, 170 and 171, which send location information for 
each print to electronic controller 136 via links 172, 173, 174 and 175, respectively. It is 
understood that cut mark sensors 168, 169, 170 and 171 could also be directly coupled to 

30 batch manager 160, scanner 124, or any other print processing device, as well as 

electronic controller 136. Indeed, in the most preferred embodiment of the present 
invention, the cut mark sensors trigger the image capture performed by scanner 124. It is 
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further understood that electronic controller 136, or a process or program residing within 
electronic controller 136, could implement the scan, cut mark sensing or batch manager 
functions. 

Since batch manager 160 maintains a continuously updated electronic 
5 record of each print's location and each print's image-quality status, the print inspection 
station 104 of this embodiment would not require sort marker 152 or any infrared light 
absorbent stickers or ink-jet printer marks to identify defective prints. Instead, batch 
manager 160 transmits the appropriate accept or reject command to packaging station 156 
at the appropriate time via link 158. In a preferred embodiment, batch manager 160 also 

1 0 sends other print-related data, such as size of the order, image size, envelope number, 

paper type, etc., to packaging station 156 or other print processing devices. In yet another 
alternative embodiment, the accept/reject notification is sent from electronic controller 
136 directly to a packaging station 1 56 via link 1 54. Finally, packaging station 1 56 sorts 
the defective print 180 into discard bin 188 and the good print 184 into keep bin 190. 

15 As would be apparent to one skilled in the art, the process of using an 

artificial neural network ("ANN") to solve problems involves two distinct phases: the 
training phase and operational phase. The training phase comprises the activities of: (1) 
building a "tr aining set" for the ANN based on a representative sample and "correct" 
results as defined by a human trainer; and (2) repeatedly exposing the training set 

20 samples to the ANN along with the correct results for each sample until the ANN has 
"learned" how to derive the correct result for each sample on its own. In reality, the 
ANN does not actually "learn" how to derive the correct result, but generates an internal 
set of mathematical rules, which, when applied to the sample inputs, yields substantially 
the same results reached by the human trainer for each of the sample inputs. 

25 At the end of the training phase, the ANN is considered to be "trained," 

which means it should be capable of solving similar, but new problems. The operational 
phase comprises exposing the "trained" ANN to new images that are similar, but not 
identical to those in the training set, and allowing the neural network to "decide" whether 
the new images meet the criteria defined by its internal rules. 

30 There are a number of general purpose ANNs, suitable for purposes of 

inspecting prints in accordance with the present invention commercially available. The 
ANN known as Trajan 4.0, manufactured and sold by Trajan Software Ltd., for example, 
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more than adequately performs the job described herein. It will be recognized by those 
with skill in the art that any multi-layer perceptron ANN with suitable back-propagation 
training software would be suitable for these purposes. 

In the preferred embodiment of the present invention, an ANN is trained to 
distinguish good prints from reject prints an abstraction of the image data, such as a set of 
histograms. In another preferred embodiment, the ANN recognizes good and bad prints 
based on raw pixel data, or a subset (or sample) of raw pixel data for each print image. In 
another embodiment, the ANN recognizes good prints and bad prints based on certain 
statistics computed from the image data, such as the maximum, minimum, average and 
10 standard deviation values of raw pixels. In these cases, specialized analog hardware, 
such as video peak detection and video signal averaging, can be utilized to provide 
certain of these statistical values in video real time for greatly enhanced accuracy. In 
another preferred embodiment, the ANN is trained to operate based on raw pixel data or a 
subset (or sample) of the raw pixel data. 
15 A more detailed description of how to train and operate an artificial neural 

network in accordance with one embodiment of the present invention is now provided. In 
the first example, the ANN is trained to distinguish between good prints and reject prints 
based on a histogram of the image data. A histogram is a data array which consists of a 
number of variables each of which represents a range of pixel values in the sampled 
20 image. The value of each variable is equal to the number of pixels in the sampled image 
with values falling within the range of pixel values assigned to that variable. In the 
preferred embodiment, a histogram is used for input to the ANN because the histogram 
accounts for all pixels in its value range, even if pixels within the value range represent 
only a very small portion of the overall image. 
25 With reference again to the flow diagram 200 depicted in FIG. 2, the first 

step for training the ANN to distinguish good prints from reject prints, is to decide how 
many points (pixels) will be sampled from each visual image and how many inputs and 
outputs are required for the ANN. See step 202 in FIG. 2. In a preferred embodiment, 
307,200 points are used, since it provides a reasonably accurate representation of the 
30 quality level of a 640 x 480 size image, which is the standard resolution for a video 
graphics array ("VGA") monitor. 
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The next step, step 204, is to decide how many inputs (also referred to 
herein as the number of "cells" or "nodes") and outputs the ANN will have. In the 
preferred embodiment, the number of inputs to the ANN is defined as the square root of 
the number of points sampled from each visual image. Thus, when the sample size is 
5 307,200 points, it works well to define an ANN with 554 inputs. In a preferred 

embodiment, a histogram is created for each cell. It is to be understood, however, that 
the present invention is not limited to the foregoing number of sample points, neural 
network inputs and histograms; alternative numbers for image sample sizes, inputs and 
histograms may be used without departing from the scope of the invention. The number 

1 0 of outputs used for the preferred embodiment of the ANN is 1 , which corresponds to the 
accept or reject decision. 

After the image sample size, number of input cells (or nodes) and outputs 
in the ANN and histograms have been determined, the next step, shown as step 206 in 
FIG. 2, is to pick a representative population of real world prints to be used in the ANN 

15 testing set. In a preferred embodiment, an adequate rule of thumb is to use approximately 
1 00 prints for each input node in the ANN. In this case, 5,540 prints would be used. It is 
understood, however, that the present invention is not limited to any particular number of 
prints used in the testing set 

The population of prints selected for the testing set should include a 

20 reasonable number of both "good" and "bad" prints. The "good" prints should be 

reasonably consistent, in terms of subject matter and image-quality, with the types of 
photographs for which consumers are willing to pay and photo processors are willing to 
charge. The "bad" prints should consist of prints having the most common types of 
defects that render photographs unmarketable, such as photographs that are blank, 

25 fogged, torn, over- or under-exposed, etc. 

The next step, depicted in flow diagram 200 as step 208, is to capture the 
visual image of each print in the population of test prints, preferably with scanner 124, 
and convert each visual image to a digital file corresponding to the visual image. In a 
step 2 1 0, histograms are created for each digital file. Then, in a step 212, the digital files 

30 are displayed to a human being, who makes a subjective determination as to whether the 
image should be "accepted" or "rejected." The effect of this determination is to tell the 
ANN what the human being considers to be the "correct" result. Next, in a step 214, the 
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histogram data, along with the human being's accept or reject decision, are assigned to a 

testing file, also known as a "training file." 

Then, the final step, step 216, in the training phase is to repeatedly execute 
the ANN on the training set until the ANN is capable of reaching the correct conclusion 
(accept or reject) on all the images in the sample set on its own. At this point the training 
of the ANN is complete and the ANN can be utilized in the print inspection system 104, 
as described above with reference to FIG. 1, to render accept/reject decisions not on 
sample or test prints, but real prints. 

With reference again to the flow diagram 200 in FIG. 2, the training steps 
for an ANN which distinguishes good prints and bad prints based on a subset of raw pixel 
data are now described. The process is very similar to the process for training an ANN to 
distinguish based on image histogram data. First, a determination is made as to the 
number of points (pixels) that will be sampled from each visual image. See step 202. As 
in the histogram example above, 307,200 points are used in the preferred embodiment, 
since it provides a reasonably accurate representation of the quality level of the entire 
image. 

Next, the number of input nodes (cells) and output nodes for the ANN is 
decided. See step 204. Again, the square root of the number of sample points is a good 
rule-of-thumb for determining the number of input nodes. Thus, the ANN will have 554 
input nodes (cells). Again, as in the histogram example, it is to be understood that the 
present invention is not limited to the foregoing number of sample points and neural 
network inputs; alternative image sample sizes and ANN input sizes may be used without 
departing from the scope of the invention. The number of outputs for the ANN is 1, 
which corresponds to the accept or reject decision. 

After the image sample size and the number of nodes in the ANN have 
been determined, the next step, shown as step 206 in FIG. 2, is to pick a representative 
population of real world prints to be used in the ANN testing set. In a preferred 
embodiment, an adequate rule of thumb is to use approximately 100 prints for each input, 
or 5,540 prints. It is also understood, however, that the present invention is not limited 
to any particular number of prints used in the testing set 

The next step, depicted in flow diagram 200 as step 208, is to capture the 
visual image of each print in the population of test prints, preferably with scanner 124, 
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and convert each visual image to a digital file corresponding to the visual image. In this 
case, no histogram is created from the digital files, so step 210 would be skipped. Next, 
in a step 212, the digital files are displayed to a human being, who makes a subjective 
determination as to whether the image should be "accepted" or "rejected," thereby telling 
the ANN what the human being considers to be the "correct" result. Next, in a step 214, 
the raw pixel data from the digital files, along with the human being's accept or reject 
decisions, are assigned to the "training file." 

Then, the final step in the training phase, step 216 in the diagram, is to 
repeatedly execute the ANN on the training set until the ANN is capable of reaching the 
correct conclusion (accept or reject) on all the images in the sample set on its own. Now 
the training of the ANN is complete and the ANN can be utilized in the print inspection 
system 104, described above with reference to FIG. 1, to render accept/reject decisions on 
real prints. 

Training an ANN to recognize defects in visual images based on statistical 
abstractions is accomplished the same way training was accomplished for histogram data 
and raw pixel data in the previous two examples, except that maximum, minimum, 
average and standard deviation values for the sample points are used as the inputs to the 
ANN instead of histograms or raw pixel data. 

With reference to the flow diagram depicted in FIGs. 3 A and 3B, a more 
complete description of a method for inspecting prints according to the present invention 
is provided. In a step 302, a length of paper 112 containing a plurality of prints 120-124 
is fed into a work path 116. In the photo processing industry, this length of paper 1 12 is 
known as a "continuous web." In a step 304, a visual image of each print in the length of 
paper 1 12 is captured, typically by a scanner 124. Scanner 124 may be implemented 
using a color sequential exposure system with a monochrome camera to lower costs, but a 
color video camera would also work well. The visual image for each print is then 
converted into a corresponding digital file in a step 306. 

In a step 308, the digital file corresponding to the visual image (and the 
print) to be inspected is selected. Then, using the artificial neural network processor 148 
described above, for example, a determination is made in step 3 10 as to whether the 
visual image corresponding to the digital file is characterized by an image-quality level 
outside a predetermined range. If the answer is "NO," then processing loops back to step 
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308, where another digital file corresponding to another visual image (and another print) 
is selected for processing. It is recognized that, in alternative embodiments, certain 
actions could be taken upon a determination that there are no defects in a print, such as 

marking the print "good." 

If the visual image is determined to be characterized by an image-quality 
level outside a predetermined range (the answer is "YES"), then processing continues in 
the flow diagram shown on FIG. 3B by way of flow connector 300B. As shown in FIG. 
3B at step 3 12, a notification of a defective print is sent to other print processing devices 
in the print inspection system 104. Some of these devices, such as the previously- 
described sort marker 152 or packaging station 156, could be disposed along the work 
path 1 16. Other devices, such as batch manager 160 or second storage medium 144, may 
not be disposed on the work path 116, but receive the information for subsequent 
processing. The notification may also be sent to still other print processing devices (not 
shown), such as a website server connected to an interconnected computer network like 
the Internet, so that a consumer or a retail clerk can log on from a remote location to 
check the image-quality status of a particular set of prints. 

If the notification is sent to a sort marker 152, then the sort marker 152 
applies an infrared light absorbent sticker or ink-jet printer mark to the defective print in a 
step 3 14. If the notification is sent to a batch manager 1 60, then, in a step 3 1 6, batch 
manager lbU update s it s cli Lu u iric^ecord-fbrT^ 

time sends a reject instruction to packaging station 156. Finally, in a step 318, packaging 
station 156 sorts the defective print 180 into reject bin 188 and the good prints 184 into 
keep bin 190. The defective prints are then ready for discarding or further processing, as 
the case may be. Step 320 determines whether there are more prints to be inspected. If 
the answer is "YES," then processing returns to step 308 by way of flowchart connector 
30A. Otherwise, processing is complete. 

With reference now to FIG. 4, a more complete description of a computer system suitable 
for use with the preferred embodiment of the present invention is provided. The 
computer system 402 includes one or more processors, such as a processor 404. The 
30 processor 404 is connected to a communication bus 406. Various software embodiments 
are described in terms of this exemplary computer system. After reading this description, 
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it will become apparent to a person skilled in the relevant art how to implement the 
invention using other computer systems and/or computer architectures. 

The computer system 402 also includes a main memory 408, preferably random 
access memory (RAM), and can also include a secondary memory 410. The secondary 
5 memory 410 can include, for example, a hard disk drive 412 and/or a removable storage 
drive 414, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, 
etc. The removable storage drive 414 reads from and/or writes to a removable storage 
unit 41 8 in a well-known manner. The removable storage unit 418, represents a floppy 
disk, magnetic tape, optical disk, etc. which is read by and written to by the removable 

1 0 storage drive 414. As will be appreciated, the removable storage unit 418 includes a 
computer usable storage medium having stored therein computer software and/or data. 

In alternative embodiments, the secondary memory 410 may include other similar 
means for allowing computer programs or other instructions to be loaded into the 
computer system 402. Such means can include, for example, a removable storage unit 

15 422 and an interface 420. Examples of such can include a program cartridge and cartridge 
interface (such as that found in video game devices), a removable memory chip (such as 
an EPROM, or PROM) and associated socket, and other removable storage units 422 and 
interfaces 420 which allow software and data to be transferred from the removable 
storage unit 422 to the computer system 402. 

20 The computer system 402 can also include a communications interface 424. The 

communications interface 424 allows software and data to be transferred between the 
computer system 402 and external devices. Examples of the communications interface 
424 can include a modem, a network interface (such as an Ethernet card), a 
communications port, a PCMCIA slot and card, etc. Software and data transferred via the 

25 communications interface 424 are in the form of signals 426 that can be electronic, 
electromagnetic, optical or other signals capable of being received by the 
communications interface 424. Signals 426 are provided to communications interface via 
a channel 428. A channel 428 carries signals 426 and can be implemented using wire or 
cable, fiber optics, a phone line, a cellular phone link, an RF link and other 

30 communications channels. 

In this document, the terms "computer program medium" and "computer usable 
medium" are used to generally refer to media such as the removable storage device 418, a 
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hard disk installed in hard disk drive 412, and signals 426. These computer program 
products are means for providing software to the computer system 402. 

Computer programs (also called computer control logic) are stored in the main 
memory 408 and/or the secondary memory 410. Computer programs can also be 
5 received via the communications interface 424. Such computer programs, when 
executed, enable the computer system 402 to perform the features of the present 
invention as discussed herein. In particular, the computer programs, when executed, 
enable the processor 404 to perform the features of the present invention. Accordingly, 
such computer programs represent controllers of the computer system 402. 

10 In an embodiment where the invention is implemented using software, the 

software may be stored in a computer program product and loaded into the computer 
system 402 using the removable storage drive 414, the hard drive 412 or the 
communications interface 424. The control logic (software), when executed by the 
processor 404, causes the processor 404 to perform the functions of the invention as 

1 5 described herein. 

In another embodiment, the invention is implemented primarily in hardware 
using, for example, hardware components such as application specific integrated circuits 
(ASICs). Implementation of such a hardware state machine so as to perform the 
functions described herein will be apparent to persons skilled in the relevant art(s). 

20 In yet another embodiment, the invention is implemented using a combination of 

both hardware and software. 

The present invention has been disclosed and described herein in what is 
considered to be its most preferred embodiments. It should be noted that variations and 
equivalents may occur to those skilled in the art upon reading the present disclosure and 

25 that such variations and equivalents are intended to come within the scope of the 
invention and the appended claims. 
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CLAIMS 
What I claim is: 

1 . A print inspection system, comprising: 

a transport mechanism for moving a length of paper along a work path, 
said length of paper containing a plurality of prints; 

a scanner positioned along said work path, said scanner being configured 
to capture a visual image of each of said plurality of prints; 

means, coupled to said scanner, for converting said visual image into a 
digital file corresponding to said visual image; and 

an electronic processor, wherein said electronic processor determines 
whether said visual image is characterized by an image-quality 
level that falls outside a predetermined image-quality range. 



2. The print inspection system of claim 1 , wherein said electronic processor is 
comprised of: 

a neural network processor that is trained to recognize a digital file having 
an image-quality level that falls outside a predetermined range of 
image-quality levels. 



3. The print inspection system of claim 1, further comprising means for storing said 
digital file on a first storage medium. 
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4. The print inspection system of claim 3, further comprising means for storing the 
result of said image-quality level determination on a second storage 
medium. 



5. The print inspection system of claim 1, further comprising: 

means, responsive to said electronic processor, for marking prints 

corresponding to visual images characterized by an image-quality 
level that falls outside said predetermined image-quality range. 



1 0 6. The print inspection system of claim 5, further comprising: 

means for discarding prints corresponding to visual images having image- 
quality levels that fall outside said predetermined image-quality 
range. 



15 7. The print inspection system of claim 1 , further comprising: 

means for tracking the current location of a print as it moves along said 
work path; and 

means for transmitting the image-quality level of said print and current 
location of that print on said work path to a print-processing 
20 device. 



8. The print inspection system of claim 1, further comprising means for identifying a 
leading and a trailing edge of each print as it moves along said work path.. 
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The print inspection system of claim 8, wherein said means for identifying 
comprises: 

a cut-marker positioned near the start of said work path, wherein said cut- 
marker inserts cut-marks on said length of paper corresponding to the 
trailing edge of each print; and 

a cut-mark sensor. 



1 0. The print inspection system of claim 7, wherein said means for tracking 

comprises: 

a cut-marker positioned near the start of said work path, wherein said cut- 
marker inserts cut-marks on said length of paper corresponding to 
the trailing edge of each print; and 

a cut-mark sensor. 

11. . A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 
capturing a visual image of each print; 

converting said visual image of each print into a digital file corresponding 
to said visual image; 

determining whether said visual image is characterized by an image- 
quality level that falls outside a predetermined image-quality 
range; and 

repeating said determining step for each digital file created in said 
converting step. 
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12. The method of claim 11, wherein said determining step is carried out using an 
electronic processor. 



5 13. The method of claim 1 1 , wherein said determining step comprises: 

training a neural network processor to recognize visual images 

characterized by an image-quality level that falls outside said 
predetermined image-quality range. 

10 

14. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that contains no 
j5 useful content. 

15. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that has a 
20 physical defect. 

16. The method of claim 13, wherein said training step comprises: 

training said neural network to recognize a visual image that has a 
25 repetitive exposure error. 



17. The method of claim 13, wherein said training step is accomplished by presenting 
said neural network processor with a histogram of the sampled visual image. 

30 



19 



D <WO 0114928A3 IA> 



WO 01/14928 PCTYUS00/22352 

1 8. The method of claim 1 3, wherein said training is accomplished by presenting said 
neural network processor with statistical abstractions of sample visual 
images. 



1 9. The method of claim 13, wherein said training comprises presenting said neural 
network processor with raw pixel data corresponding to sample visual 
images. 



20. The method of claim 1 3, wherein said training is accomplished by presenting said 
neural network processor with a subset of raw pixel data corresponding to 
sample visual images. 



2 1 . The method of claim 1 1, further comprising the step of storing said digital file on 
a first storage medium. 



22. The method of claim 1 1, further comprising the steps of: 

marking prints corresponding to visual images characterized by an image- 
quality level that falls outside said predetermined image-quality 
range to form marked prints; and 

discarding said marked prints containing said markings. 



23. The method of claim 1 1, further comprising the step of tracking the current 
location of each print as it moves along said work path. 
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The method of claim 23, further comprising the step of: 

transmitting image-quality level of a print and current location of that print 
on said work path to a print-processing device. 



25. The method of claim 24, wherein said image-quality level and current location are 
transmitted to said print-processing device via a link to an interconnected 
computer network. 

26. A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 

marking a leading or trailing edge of each print with cut-marks; 

sensing the location of said cut-marks as said length of paper moves along 
said work path; and 

in response to said sensing, 

capturing a visual image of each print, 

converting said visual image into a digital file corresponding to 
said visual image, and 

determining whether said visual image is characterized by an 
image-quality level that falls outside a predetermined 
image-quality range. 



The method of claim 26, wherein said determining step is carried out using 
electronic processor. 
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28. The method of claim 26, wherein said determining step comprises: 

training a neural network processor to recognize visual images 

characterized by an image-quality level that falls outside said 
predetermined image-quality range. 

29. A method for inspecting prints, comprising the steps of: 

feeding into a work path a length of paper containing a plurality of prints; 
capturing a visual image of each print; 

converting said visual image of each print into a digital file corresponding 
to said visual image; 

determining via an electronic processor whether said visual image is 
characterized by an image-quality level that falls outside a 
predetermined image-quality range; and 

repeating said determining step for each digital file created in said 
converting step. 
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DETERMINE NUMBER OF POINTS TO BE SAMPLED 
FROM EACH VISUAL IMAGE 



DETERMINE THE NUMBER OF INPUTS (NODES) 
AND OUTPUTS THE ANN WILL HAVE 



PICK A REPRESENTATIVE SET OF REAL-WORLD PRINTS 
TO BE USED IN THE ANN TESTING SET 



CAPTURE THE VISUAL IMAGES OF EACH PRINT AND 
CONVERT EACH VISUAL IMAGE TO A DIGITAL FILE 



CREATE HISTOGRAMS FOR EACH DIGITAL FILE 
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DISPLAY THE VISUAL IMAGE CORRESPONDING TO 
EACH DIGITAL FILE TO A HUMAN BEING WHO 
DETERMINES WHETHER THE IMAGE SHOULD BE 
ACCEPTED OR REJECTED 



-212 



ASSIGN THE ACCEPT/REJECT DECISION AND DIGITAL 
FILES TO A TRAINING FILE 
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REPEATEDLY PROCESS THE TRAINING FILE THROUGH 
THE ANN UNTIL THE ANN IS CAPABLE OF 
GENERATING THE SAME ACCEPT OR REJECT RESULT 
AS HUMAN BEING 
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FEED PAPER CONTAINING A 
PLURALITY OF PRINTS INTO A WORKPATH 
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IS THE VISUAL IMAGE CHARACTERIZED 
BY AN IMAGE QUALITY LEVEL OUTSIDE 
A PREDETERMINED RANGE? 
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